In a data or computing network, traffic anomaly detection is a major concern. Traffic anomalies include unusual and significant changes in a network's traffic levels, which can often span multiple links and nodes. Diagnosing traffic anomalies is critical for both network operators and end users. It is a difficult problem because one must extract and interpret anomalous patterns from large amounts of high-dimensional, noisy data, as traffic variation is large by nature.
Understanding the nature of traffic anomalies in a network is important for at least two reasons, regardless of whether a traffic anomaly is malicious or unintentional:
(a). Traffic anomalies can create congestion in a network and stress resource utilization of network devices (e.g., routers or switches), thus it is critical to detect traffic anomalies from an operational standpoint;
(b). Traffic anomalies can have a drastic impact on a customer or an end user even (e.g., service down due to misconfiguration of network devices) even if it does not necessary impact the network.
A significant problem in diagnosing traffic anomalies is that its formation and cause can vary considerably: from Denial of Service (DoS) attack, to router misconfiguration, to the results of network device policy modification (e.g., border gateway protocol (BGP) policy changes), and etc. For example, DoS attack occurs when a large amount of traffic sent from one or more hosts consumes a large amount of resources in the network such as a link or a web server. This artificially increased high load denies (prevents) services to legitimate users of that resource. Despite many academic proposals in this area, today's Internet still has few protection mechanisms to prevent such attacks. Moreover, distributed DoS attack (DDoS) is even more dangerous. DDoS attack can also target at the network infrastructure beyond the individual web servers.
In order to identify traffic anomalies, network and system administrators have begun to deploy automated response systems to look for anomalous behaviors that might be an attack. However, these automated response systems can be difficult to deploy, partially because the lack of support from the commercial router/switch vendors. Also they often are heavy weighted, meaning that they require capturing a large amount of traffic in the network and thus introduce a large overhead to both the network management system and the network itself. A better way to detect traffic anomalies is needed.